A compositional framework for Markov processes
نویسندگان
چکیده
منابع مشابه
A Compositional Framework for Markov Processes
We define the concept of an ‘open’ Markov process, or more precisely, continuous-time Markov chain, which is one where probability can flow in or out of certain states called ‘inputs’ and ‘outputs’. One can build up a Markov process from smaller open pieces. This process is formalized by making open Markov processes into the morphisms of a dagger compact category. We show that the behavior of a...
متن کاملCompositional Reasoning for Markov Decision Processes
Markov decision processes (MDPs) have long been used to model qualitative aspects of systems in the presence of uncertainty. However, much of the literature on MDPs takes a monolithic approach, by modelling a system as a particular MDP; properties of the system are then inferred by analysis of that particular MDP. In this paper we develop compositional methods for reasoning about the qualitativ...
متن کاملCompositional Reasoning for Interval Markov Decision Processes
Model checking probabilistic CTL properties of Markov decision processes with convex uncertainties has been recently investigated by Puggelli et al. Such model checking algorithms typically suffer from the state space explosion. In this paper, we address probabilistic bisimulation to reduce the size of such an MDP while preserving the probabilistic CTL properties it satisfies. In particular, we...
متن کاملCompositional reasoning for weighted Markov decision processes
Weighted Markov decision processes (MDPs) have long been used to model quantitative aspects of systems in the presence of uncertainty. However, much of the literature on such MDPs takes a monolithic approach, by modelling a system as a particular MDP; properties of the system are then inferred by analysis of that particular MDP. In contrast in this paper we develop compositional methods for rea...
متن کاملThe compositional construction of Markov processes
The idea of this paper is to introduce into the algebra of automata introduced in [4] probabilities on the actions. This permits a compositional description of probabilistic processes, in particular of Markov chains, and for this reason we call the automata we introduce Markov automata. We define a Markov automaton with a given set A of “signals on the left interface”, and set B of “signals on ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Mathematical Physics
سال: 2016
ISSN: 0022-2488,1089-7658
DOI: 10.1063/1.4941578